Automated drowsiness detection for improved driving safety

Several approaches were proposed for the detection and prediction of drowsiness. The approaches can be categorized as estimating the fitness of duty, modeling the sleep-wake rhythms, measuring the vehicle based performance and online operator monitoring. Computer vision based online operator monitoring approach has become prominent due to its predictive ability of detecting drowsiness. Previous studies with this approach detect driver drowsiness primarily by making preassumptions about the relevant behavior, focusing on blink rate, eye closure, and yawning. Here we employ machine learning to datamine actual human behavior during drowsiness episodes. Automatic classifiers for 30 facial actions from the Facial Action Coding system were developed using machine learning on a separate database of spontaneous expressions. These facial actions include blinking and yawn motions, as well as a number of other facial movements. In addition, head motion was collected through automatic eye tracking and an accelerometer. These measures were passed to learning-based classifiers such as Adaboost and multinomial ridge regression. The system was able to predict sleep and crash episodes during a driving computer game with 96% accuracy within subjects and above 90% accuracy across subjects. This is the highest prediction rate reported to date for detecting real drowsiness. Moreover, the analysis revealed new information about human behavior during drowsy drivin

Discrimination of moderate and acute drowsiness based on spontaneous facial expressions

It is important for drowsiness detection systems to identify different levels of drowsiness and respond appropriately at each level. This study explores how to discriminate moderate from acute drowsiness by applying computer vision techniques to the human face. In our previous study, spontaneous facial expressions measured through computer vision techniques were used as an indicator to discriminate alert from acutely drowsy episodes. In this study we are exploring which facial muscle movements are predictive of moderate and acute drowsiness. The effect of temporal dynamics of action units on prediction performances is explored by capturing temporal dynamics using an overcomplete representation of temporal Gabor Filters. In the final system we perform feature selection to build a classifier that can discriminate moderate drowsy from acute drowsy episodes. The system achieves a classification rate of .96 A’ in discriminating moderately drowsy versus acutely drowsy episodes. Moreover the study reveals new information in facial behavior occurring during different stages of drowsiness

Real Time Face Detection and Facial Expression Recognition: Development and Applications to Human Computer Interaction

to human computer interaction and force us to think in new ways about how computers could be used in daily life. Face to face communication is a real-time process operating at a a time scale in the order of 40 milliseconds. The level of uncertainty at this time scale is considerable, making it necessary for humans and machines to rely on sensory rich perceptual primitives rather than slow symbolic inference processes. In this paper we present progress on one such perceptual primitive. The system automatically detects frontal faces in the video stream and codes them with respect to 7 dimensions in real time: neutral, anger, disgust, fear, joy, sadness, surprise. The face finder employs a cascade of feature detectors trained with boosting techniques [16, 3]. The expression recognizer receives image patches located by the face detector. A Gabor representation of the patch is formed and then processed by a bank of SVM classifiers. A novel combination of Adaboost and SVM's enhances performance. The system was tested on the Cohn-Kanade dataset of posed facial expressions [7]. The generalization performance to new subjects for a 7way forced choice correct. Most interestingly the outputs of the classifier change smoothly as a function of time, providing a potentially valuable representation to code facial expression dynamics in a fully automatic and unobtrusive manner. The system has been deployed on a wide variety of platforms including Sony's Aibo pet robot, ATR's RoboVie, and CU animator, and is currently being evaluated for applications including automatic reading tutors, assessment of human-robot interaction

Fully automatic coding of basic expressions from video

We present results on a user independent fully automatic system for real time recognition of basic emotional expressions from video. The system automatically detects frontal faces in the video stream and codes them with respect to 7 dimensions: neutral, anger, disgust, fear, joy, sadness, surprise. The face finder is based on [18] with a more complex feature space and multiframe exclusion rules. The expression recognizer receives image patches located by the face detector. A Gabor representation [2] of the patch is formed and processed by bank of 63 SVMs [3]. The final coding into 7 expression categories is performed via multinomial ridge logistic regression, a natural generalization of SVMs to the multinomial case. Strategies for performing multiclass decisions using SVM’s are compared. The effectiveness of Gabor magnitude filters is examined. Different methods for combining information from the upper and lower regions of the face are also discussed. Results on the Cohn-Kanade dataset of posed facial expressions are discussed [9]. The generalization performance to novel subjects on 7-way forced choice based on 614 frames was 91.5 % correct. Most interestingly the outputs of the classifier change smoothly as a function of time, providing a potentially valuable representation to code facial expression dynamics in a fully automatic and unobtrusive manner.

A Discriminative Parts Based Model Approach for Fiducial Points Free and Shape Constrained Head Pose Normalisation in the Wild

This paper proposes a method for parts- based view-invariant head pose normalisation, which works well even in difficult real- world conditions. Handling pose is a classi-cal problem in facial analysis. Recently, parts- based mod-els have shown promising performance for facial landmark points detection ‘in the wild’. Leveraging on the success of these models, the proposed data- driven regression frame-work computes a constrained normalised virtual frontal head pose. The response maps of a discriminatively trained part detector are used as texture information. These sparse texture maps are projected from non- frontal to frontal pose using block- wise structured regression. Finally, a facial kinematic shape constraint is achieved by applying a shape model. The advantages of the proposed approach are: a) no explicit dependence on the outputs of a facial parts detector and, thus, avoiding any error propagation owing to their failure; (b) the application of a shape prior on the recon-structed frontal maps provides an anatomically constrained facial shape; and c) modelling head pose as a mixture- of-parts model allows the framework to work without any prior pose information. Experiments are performed on the Multi-PIE and the ‘in the wild ’ SFEW databases. The results demonstrate the effectiveness of the proposed method. 1